A distributed processing system comprises a shared disk file system operating on more than one computer. Each of the computers (or clients) in the distributed processing system comprises an instance of an operating system. Each of the clients is coupled for parallel data-sharing access to files residing on storage in the form of network attached disks or storage servers. A user in the form of a human or an application accesses the storage through one or more clients.
A distributed processing system (also referred to as a storage system) manages the resources contained by the storage system to provide adequate service to applications and clients. Many storage systems have mechanisms that ensure that applications or clients acting on behalf of the application receive the performance that the application needs to perform properly. Placing an upper bound on performance ensures that one application cannot use all the resource in the storage system and cause unacceptable performance degradation for other applications.
Conventional methods of enforcing an upper bound on performance comprise associating performance limits with data items or using sessions between an application and a storage device in the storage system. In associating performance limits with data items, the storage system records that a particular data item has a performance limit. The data item can be, for example, an object, a partition, a logical unit number (LUN), a file, etc. The performance limit is applied to all accesses to the item, regardless of the application or client issuing a request. When using sessions, the application or client negotiates a session comprising performance limits with the storage device. The session is often related to a network transport connection. A maximum service level is associated with the session and the negotiated performance limits are applied to all IO requests issued in the context of the session, regardless of the data item.
These conventional methods provide local enforcement of a maximum service level at a storage device; no other component is involved in the request-by-request decisions that are made. The mechanisms of these conventional methods can further be built such that clients or applications cannot subvert the mechanisms even if the clients or applications misbehave. Although these conventional methods have proven to be useful, it would be desirable to present additional improvements.
In a distributed storage system, management of resources becomes difficult. Resource management of storage units or storage servers in a distributed storage system can be centralized in a central management server. However, the resulting processing load on the central management server imposes a limit on performance of the distributed storage system and further limits the largest scale the distributed storage system can achieve.
Otherwise, resource management can be decentralized such that, for example, each of the storage servers in the distributed storage system manages resources individually. With decentralized resource management, performance of the distributed storage system can scale with the size of the distributed storage system. However, consistent global policies are difficult to ensure in a distributed storage system with decentralized resource management.
Furthermore, performance requirements of a centralized storage system may vary depending on the usage of the centralized storage system. In some environments, the centralized storage system may be required to guarantee a minimum level of service or quality of service. For example, users of business applications expect at most a particular response time or at least a certain transaction throughput. In other environments, the centralized storage system may be required to limit an amount of resources consumed by an application or a user. For example, users in scientific or departmental computing share storage with no one user allowed to consume more than a predetermined share. Some applications are only concerned with storage capacity; other applications require control over performance as well.
Conventional file systems, including distributed file systems, often provide capacity quotas that impose a maximum capacity usage on any one user. However, these file systems do not guarantee to a user a minimum amount of capacity. Further, file systems usually do not consider performance-oriented resources.
Conventional distributed logical volume managers and storage virtualization engines provide aggregation of capacity from multiple storage devices into one logical volume. This aggregation of capacity provides guaranteed capacity to a user without allowing for large-scale over-commitment of capacity in the manner of file systems.
Some conventional virtualization engines support IO throttling mechanisms that try to ensure a given level of performance for an application. However, these virtualization engines do not reserve resources for an application even when that application is not active. Reservation of resources is required so that when an application tries to create an IO session with a particular performance level, the admission of the session can be guaranteed.
Conventional file systems, volume managers, and storage virtualization engines do not distribute resource control work to the storage devices. These conventional approaches all use passive storage devices that only read and write data blocks from one to another. Furthermore, conventional systems open a session by opening a connection to a specific server, limiting the session to one server. To migrate to a new server, the conventional approach requires closing a current session and opening a new session. While migrating to a new server, conventional approaches may be required to wait until sufficient resources become available for the migrating session.
Some conventional distributed storage systems aggregate multiple object storage devices into a logical object storage device. The object storage devices in this approach comprise some local intelligence and make local resource allocation decisions such as determining which blocks should hold the data of an object. Although this technology has proven to be useful, it would be desirable to present additional improvements. This conventional system enforces maximum limits in the form of quotas on storage capacity. The object storage device can maintain quotas on partitions; these quotas on partitions can be used to build distributed enforcement of quota across multiple storage devices by ensuring that a sum of the quotas on partitions is no larger than the allowed distributed quota.
Conventional sessions in a virtual resource pool provide fine-grained inter-application isolation. However, these sessions are typically negotiated between one client and the storage device; a conventional virtual resource pool manager does not have an opportunity to effect policies that involve multiple storage devices.
What is therefore needed is a system, a service, a computer program product, and an associated method for managing resources in a distributed storage system that allows a virtual resource pool manager to determine service levels for applications while ensuring that applications that share a data item are isolated in performance from each other. Furthermore, by having a virtual resource pool manager determine a service level, the sum of all service levels for a client, application, or user can be compared to an overall performance limit. Also needed is a solution that allows a variety of performance levels within one session while maintaining performance isolation. The need for such a solution has heretofore remained unsatisfied.